Cooling system

ABSTRACT

A system and method for cooling a room configured to house a plurality of computer systems. A heat exchanger unit is configured to receive air from the room and to deliver air to the room. The heat exchanger unit is supplied with cooling fluid operable to cool the received air in the heat exchanger unit. At least one of the temperature of the cooling fluid supplied to the heat exchanger unit and the air delivery to the room may be controlled in response to temperatures sensed at one or more locations in the room.

CROSS-REFERENCES

[0001] The present invention is related to pending U.S. application Ser.No. 09/970,707, filed Oct. 5, 2001, and entitled “SMART COOLING OF DATACENTERS”, by Patel et al., U.S. application Ser. No. 10/157,892 filed onMay 31, 2002 (HP Docket No. 100200782-1) entitled “CONTROLLED COOLING OFA DATA CENTER”, by Patel et al., U.S. application Ser. No. 10/122,010,filed on Apr. 24, 2002, and entitled, “DATA CENTER ENERGY MANAGEMENTSYSTEM”, by Friedrich et al., all of which are assigned to the assigneeof the present invention and are incorporated by reference herein intheir entireties.

FIELD OF THE INVENTION

[0002] This invention relates generally to cooling systems.

BACKGROUND OF THE INVENTION

[0003] A data center may be defined as a location, e.g., room, thathouses computer systems arranged in a number of racks. A standard rackmay be defined as an Electronics Industry Association (EIA) enclosure,78 in. (2 meters) wide, 24 in. (0.61 meter) wide and 30 in. (0.76 meter)deep. Standard racks may be configured to house a number of computersystems, e.g., about forty (40) systems, with future configurations ofracks being designed to accommodate up to eighty (80) systems. Thecomputer systems typically include a number of components, e.g., one ormore of printed circuit boards (PCBs), mass storage devices, powersupplies, processors, micro-controllers, semi-conductor devices, and thelike, that may dissipate relatively significant amounts of heat duringthe operation of the respective components. For example, a typicalcomputer system comprising multiple microprocessors may dissipateapproximately 250 W of power. Thus, a rack containing forty (40)computer systems of this type may dissipate approximately 10 KW ofpower.

[0004] The power required to transfer the heat dissipated by thecomponents in the racks to the cool air contained in the data center isgenerally equal to about 10 percent of the power needed to operate thecomponents. However, the power required to remove the heat dissipated bya plurality of racks in a data center is generally equal to about 50percent of the power needed to operate the components in the racks. Thedisparity in the amount of power required to dissipate the various heatloads between racks and data centers stems from, for example, theadditional thermodynamic work needed in the data center to cool the air.In one respect, racks are typically cooled with fans that operate tomove cooling fluid, e.g., air, cooling fluid, etc., across the heatdissipating components; whereas, data centers often implement reversepower cycles to cool heated return air. The additional work required toachieve the temperature reduction, in addition to the work associatedwith moving the cooling fluid in the data center and the condenser,often add up to the 50 percent power requirement. As such, the coolingof data centers presents problems in addition to those faced with thecooling of the racks.

[0005] Conventional data centers are typically cooled by operation ofone or more air conditioning units. For example, compressors of airconditioning units typically require a minimum of about thirty (30)percent of the required operating energy to sufficiently cool the datacenters. The other components, e.g., condensers, air movers (fans),etc., typically require an additional twenty (20) percent of therequired cooling capacity. As an example, a high density data centerwith 100 racks, each rack having a maximum power dissipation of 10 KW,generally requires 1 MW of cooling capacity. Air conditioning units witha capacity of 1 MW of heat removal generally requires a minimum of 300KW input compressor power in addition to the power needed to drive theair moving devices, e.g., fans, blowers, etc. Conventional data centerair conditioning units do not vary their cooling fluid output based onthe distributed needs of the data center. Instead, these airconditioning units generally operate at or near a maximum compressorpower even when the heat load is reduced inside the data center.

[0006] The substantially continuous operation of the air conditioningunits is generally designed to operate according to a worst-casescenario. For example, air conditioning systems are typically designedaround the maximum capacity and redundancies are utilized so that thedata center may remain on-line on a substantially continual basis.However, the computer systems in the data center may only utilize around30-50% of the maximum cooling capacity. In this respect, conventionalcooling systems often attempt to cool components that may not beoperating at a level which may cause their temperatures to exceed apredetermined temperature range. Consequently, conventional coolingsystems often incur greater amounts of operating expenses than may benecessary to sufficiently cool the heat generating components containedin the racks of data centers.

SUMMARY OF THE INVENTION

[0007] According to one embodiment, the present invention pertains to amethod for cooling a room configured to house a plurality of computersystems. In the method, there is provided a heat exchanger unitconfigured to receive air from the room and to deliver air to the room.The heat exchanger unit is supplied with cooling fluid that is operableto cool the received air in the heat exchanger unit. In addition,temperatures at one or more locations in the room are sensed and atleast one of the temperature of the cooling fluid and the air deliveryto the room is controlled in response to the sensed temperatures at theone or more locations.

[0008] According to another embodiment, the invention relates a systemfor cooling a room containing one or more computer systems. The systemincludes a heat exchanger unit configured to receive cooling fluidthrough a fluid line and to receive air, where the air may be cooledthrough heat transfer with the cooling fluid. The heat exchanger unithas at least one fan configured to cause air to flow out of the heatexchanger unit. The system also includes a heat exchanger controlleroperable to control a supply of the cooling fluid to the heat exchangerunit and operable to control the speed of the at least one fan. Thesystem further includes a device for cooling the cooling fluid and acooling device controller configured to operate the device for coolingto vary the temperature of the cooling fluid.

[0009] According to yet another embodiment, the present inventionpertains to a system for cooling computer systems housed in one or moreracks maintained in a room. The system includes means for cooling airincluding means for receiving cooling fluid. The system also includesmeans for delivering cooled air to the computer systems and means formeasuring temperatures at one or more locations in the room.Additionally, the system includes means for controlling delivery of thecooled air in response to the temperature measurements and means forcontrolling the temperature of the cooling fluid.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Features of the present invention will become apparent to thoseskilled in the art from the following description with reference to thefigures, in which:

[0011]FIG. 1 shows a simplified schematic illustration of a roomcontaining a cooling system in accordance with an embodiment of theinvention;

[0012]FIG. 2 is an enlarged partly cross-sectional view of section A inFIG. 1;

[0013]FIG. 3 is a schematic illustration taken along lines III-III ofFIG. 1;

[0014]FIG. 4 is an exemplary block diagram for a cooling systemaccording to an embodiment of the invention; and

[0015]FIG. 5 shows a flow diagram of an operational mode according to anembodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0016] For simplicity and illustrative purposes, the present inventionis described by referring mainly to an exemplary embodiment thereof. Inthe following description, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent however, to one of ordinary skill in the art, that thepresent invention may be practiced without limitation to these specificdetails. In other instances, well known methods and structures have notbeen described in detail so as not to unnecessarily obscure the presentinvention.

[0017] According to an embodiment of the present invention, a coolingsystem may include a heat exchanger unit (HEU) configured to receivecooling fluid (e.g., chilled water, R134 a, ethylene glycol mixture, andthe like). The HEU may be designed to cause air to exchange heat withthe cooling fluid to thereby cool the air. The HEU may also be designedto deliver the cooled air to one or more computer systems. The coolingsystem may also include a cooling device operable to cool the coolingfluid. The cooling device may comprise one or more of known coolingsystems (e.g., a closed loop refrigeration cycle configured to transferheat with the cooling fluid). The cooling device may therefore include asecond generally closed-loop system having refrigerant flowing through arefrigerant line and positioned to receive heat from the cooling fluid.

[0018] One or more controllers may be provided to operate the coolingsystem components. The one or more controllers may be configured tomonitor environmental conditions within the room, and more particularlyin areas around predetermined computer systems, and manipulate operationof the HEU and the cooling device according to the monitoredenvironmental conditions. For example, the one or more controllers maycontrol the temperature of the cooling fluid and/or the output of theHEU.

[0019] In one respect, by manipulating the output of the HEU and thetemperature of the cooling fluid, the amount of energy required to coolthe computer systems may be relatively low. Thus, according to theembodiment described above, instead of operating components of thecooling device, e.g., compressors, heat exchangers, fans, etc., atsubstantially 100 percent of the anticipated heat dissipation from thecomputer systems, those components may be operated according to theactual cooling needs. In addition, the computer systems and the racks inwhich they are housed may be positioned throughout the room according totheir anticipated heat loads, to additionally increase control over theenergy required by the cooling system. Furthermore, as described in U.S.application Ser. No. 10/122,010, filed on Apr. 24, 2002, the loads maybe distributed between various computer systems to further increaseenergy efficiency of air conditioning resources.

[0020] In another respect, the positioning of the racks may bedetermined through implementation of numerical modeling and metrology ofthe cooling fluid flow throughout the data center. In addition, thenumerical modeling may be implemented to determine the volume flow rateand velocity of the cooling fluid flow through each of the HEU's.

[0021] A more detailed description of the above-described embodiment maybe found in co-pending U.S. application Ser. No. 09/970,707, filed Oct.5, 2001, which is assigned to the assignee of the present invention andis incorporated by reference herein in its entirety.

[0022] Referring first to FIG. 1, there is illustrated a room 10, e.g.,a data center, housing a plurality of racks 12, and a cooling system 20.Also illustrated is a raised floor 14 that may be provided to enableplacement of wires, tubes, and the like for delivery into and from theracks 12.

[0023] The racks 12 may generally house a plurality of components (notshown), e.g., processors, micro-controllers, memories, semi-conductordevices, and the like. The components may be elements of a plurality ofsubsystems (not shown), e.g., computers, servers, etc. The subsystemsand the components may be implemented to perform various electronic,e.g., computing, switching, routing, displaying, and the like,functions. In the performance of these electronic functions, thecomponents, and therefore the subsystems, may dissipate relatively largeamounts of heat. Because the racks 12 have been generally known toinclude upwards of forty (40) or more subsystems, they may requiresubstantially large amounts of cooling fluid to maintain the subsystemsand the components generally within a predetermined operatingtemperature range. According to an embodiment of the invention, bysubstantially controlling the amount of cooling air delivered to thecomponents and the subsystems located in the racks 12 based upon theirrespective heat loads, the power consumed by the cooling system 20 tocool and supply the cooling fluid may also be controlled.

[0024] The cooling system 20 is illustrated as including a plurality ofHEU's 22 and a cooling device 24. The HEU's 22 may be supported from aceiling 16 of the room 10 by one or more supports 26. The HEU's 22 maycomprise any reasonably suitable air conditioning unit designed toreceive air and to deliver the received air, e.g., the DATACOOLenvironmental control system manufactured and distributed by Liebert ofIrvine, Calif. As can be seen in FIG. 1, the HEU's 22 are positioned toreceive air that may become heated by virtue of the heat dissipated by,for example, the computer systems located in the racks 12. Although anHEU 22 is illustrated as positioned over a rack 12, it should beunderstood that the HEU's 22 may be positioned at various otherlocations. For example a plurality of HEU's 22 may be positioned to coola single rack. Determinations of HEU 22 placement within the room 10 maybe predicated upon techniques designed to optimize cooling performanceand/or energy efficiency.

[0025] The HEU's 22 may be positioned to generally supply the racks 12with cooling air. More particularly, the HEU's 22 may be positioned tosupply cooling air to the racks 12 in accordance with their heatdissipation. By way of example, computer systems that undergo higherloads are known to generate greater amounts of heat as compared tocomputer systems that undergo lower or no loads. Thus, the HEU's 22 maybe positioned and operated to supply those computer systems (and racks12) that generate greater amounts of heat with greater mass flow ratesof cooling air and/or relatively lower temperature air. In addition,those computer systems (and racks 12) that generate little or no heat,may likewise, receive little or no cooling air and/or relatively highertemperature air. In this respect, the amount of energy required tooperate the HEU's 22 specifically, and the cooling system 20 as a whole,may be substantially optimized according to the demands of the computersystems.

[0026] Alternatively, or additionally to the above, because usage of thecomputer systems contained in the racks 12 may vary at different timesof the day, instead of varying the position of the HEU's 22, the loadplaced on the computer systems may be varied. For instance, as describedin co-pending U.S. application Ser. No. 10/122,010, the work load onsome of the computer systems may be performed by other computer systemsto substantially maximize energy efficiency of the cooling system. Inthis regard, the load may be transferred, for example, to one locationin the room 10 if it is determined that such a load transfer is moreenergy efficient than under normal operating conditions. Moreover, theload transfer may occur at different times of the day and as loadrequirements vary.

[0027] The cooling device 24 may comprise any reasonably suitable typeof cooling device designed to adequately cool the cooling fluid. Inaddition, the cooling device 24 may include the capability of varyingthe temperature of the cooling fluid. Some suitable cooling devices mayinclude those that implement air conditioners, heat exchangers, heatpumps, variable capacity chillers, evaporative cooling systems, and thelike. For example, the cooling device 24 may comprise a closed-looprefrigeration cycle apparatus having a heat transfer section where theheat from the cooling fluid in a fluid line 28 may be transferred torefrigerant contained in the closed-loop refrigeration cycle apparatus.

[0028] Although the cooling device 24 is illustrated as being locatedoutside of the room 10, it should be understood that the cooling device24 may be positioned within the room 24 without deviating from the scopeof the present invention.

[0029] The cooling fluid may be configured to flow through the HEU's 22and return to the cooling device 24 via fluid lines 28. As seen in FIG.1, the fluid line 28 generally forms a closed loop system in which thecooling fluid may become heated in the HEU's 22 and cooled in thecooling device 24.

[0030] Although one room 10 is illustrated in FIG. 1, it should beunderstood that the room 10 may comprise more than one room and that thecooling system 20 may be configured to cool a plurality of rooms.

[0031] With reference now to FIG. 2, there is illustrated an enlargedpartly cross-sectional view of section A in FIG. 1. As seen in FIG. 2,the HEU 22 generally comprises, among other things, a plurality of fans30 and an opening 32. The fans 30 are designed to cause air from theroom 10 to flow into the opening 32, as indicated by arrows 34. The fans30 are also configured to cause the air to flow back out into the room10, in the manner indicated by arrows 36. It should be understood thatthe HEU 22 may include additional components and that some of thecomponents described may be removed and/or modified without departingfrom the scope of the invention.

[0032] The fluid line 28 is situated to run through the HEU 22 below thefans 30 and in the direction of air flow from the HEU 22 to the room 10.In this respect, as the air flows by the fluid line 28, heat may betransferred from the air to the cooling fluid contained in the fluidline 28. Consequently, the air flowing out of the HEU 22 may be at alower temperature relative to the air entering the HEU 22.

[0033] A pump 38 may be provided along the fluid line 28 at a locationgenerally upstream of the HEU 22. The pump 38 may be designed to applysufficient pressure to the cooling fluid entering the HEU 22 to enablethe cooling fluid to flow at a predetermined rate through the HEU 22. Inthis respect, the pump 38 may comprise any pump capable of performingthe above-described operations in a reasonably suitable manner. The pump38 may be configured to supply cooling fluid to a plurality of HEU's 22and to control the pressure of the cooling fluid flowing through one ormore of the HEU's 22. A valve 40 is illustrated between the pump 38 andthe HEU 22 along the fluid line 28. The valve 40 may be anelectronically controllable valve capable of varying the flow of coolingfluid through the fluid line 28, as well as substantially stopping theflow of the cooling fluid. As a result, the volume flow rate of thecooling fluid through the fluid line 28 extending through the HEU 22 maybe manipulated at predetermined levels such that the temperature of theair flowing out of the HEU 22 may be substantially optimized.

[0034] Although the pump 38 and the valve 40 are illustrated as separatecomponents, it should be understood that their respective functions maybe performed by a single component, e.g., the pump 38. In this example,the pump 38 may be configured as a variable speed pump configured tovary the flow of cooling fluid through the HEU 22.

[0035] A pipe 42 may be located along the fluid line 28 at a locationdownstream of the HEU 22. The pipe 42 may enable cooling fluid that haspassed through the HEU 22 to be introduced into cooling fluid enteringinto the HEU 22. A pipe valve 44 may be positioned along the pipe 42 tosubstantially control the amount of this cooling fluid introduced intothe cooling fluid entering into the HEU 22. The pipe valve 44 may be anyreasonably suitable valve (e.g., a three-way valve) capable ofcontrolling the cooling fluid flow through the pipe 42.

[0036] In this respect, the temperature of the cooling fluid may thus befurther controlled. By way of example, the temperature of the coolingfluid may be increased prior to its introduction into an HEU 22 in amulti-HEU system (see e.g., FIG. 3). In this type of configuration, whenone rack 12 is dissipating a greater amount of heat relative to anotherrack 12, the temperature of cooling air flow supplied to each of theseracks may differ from one another. That is, the rack 12 dissipating thegreater amount of heat may receive air flow from a first HEU 22 that isat a lower temperature compared to the rack that is dissipating a lesseramount of heat. Consequently, the temperature of the cooling fluidintroduced into that first HEU may be at a lower temperature than thecooling fluid introduced into a second HEU.

[0037] Temperature sensors 46 may be located in a supply portion and areturn portion of the fluid line 28 such that a determination may bemade as to changes in the cooling fluid temperature prior to andfollowing flow through the HEU 22. This information may be implementedto determine the amount of return cooling fluid to be introduced intothe supply portion of the fluid line 28.

[0038] A temperature sensor 48 e.g., a thermocouple or the like, isillustrated on the rack 12. A single temperature sensor 48 is depictedin FIG. 2 for purposes of simplicity, it should, however, be understoodthat a plurality of temperature sensors 48 may be positioned at variouslocations of the rack 12 as well as other areas of the room 12. Thetemperature sensor 48 may supply temperature readings to a controller(not shown) configured to operate the HEU 22, the pump 38, the valve 40,and the pipe valve 44. In addition, the temperature readings may also besupplied to a controller (not shown) configured to operate the coolingdevice 24. The manner of information transmittal between the temperaturesensor 48 and the controllers may comprise any reasonably suitablearrangement, e.g., wired or wireless connection.

[0039] In place of, or in addition to, the temperature sensor 48, amobile device (not shown) designed to gather or measure at least onelocal environmental condition (e.g., temperature, air flow, humidity,etc.) in the room 10 may be implemented. By way of example, a suitablemobile device may comprise the environmental condition detecting devicedescribed in co-pending U.S. application Ser. No. 10/157,892, filed onMay 31, 2002 (HP Docket No. 100200782-1) entitled “CONTROLLED COOLING OFA DATA CENTER”, by Patel et al., which is assigned to the assignee ofthe present invention and incorporated by reference herein in itsentirety. Information gathered by the mobile device may be transmittedto the controller, which may vary operation of the cooling system 20 inresponse to the information.

[0040] Alternatively, or in addition to, the above-describedenvironmental condition sensing devices, the amount of heat generated bythe computer systems contained in the racks 12, and the resultingnecessary cooling requirements, may be anticipated according to theamount of load placed on the computer systems. A suitable method ofimplementing anticipated sensing is described in co-pending U.S.application Ser. No. 09/970,707, which is assigned to the assignee ofthe present invention and incorporated by reference herein in itsentirety.

[0041] Referring now to FIG. 3, there is shown a schematic illustrationtaken along lines III-III of FIG. 1. FIG. 3 illustrates a plurality ofHEU's 22 and a plurality of fluid lines 28 emanating from a pair ofpumps 38. The illustration also shows that the HEU's 22 include rows offans 30 located adjacent to the openings 32. Situated beneath each ofthe fans 30 are fluid lines 28. In addition, located generally upstreamof the HEU's 22 are respective valves 40, which may comprise three-wayvalves. As can be seen in FIG. 3, the pumps 38 may control the supply ofcooling fluidt hrough a plurality of HEU's 22. In addition, it may beseen that the valves 40 generally enable control of cooling fluid flowthrough respective fluid lines 28.

[0042] In this respect, the temperature of the air flowing out of theHEU's 22, and more specifically, out of each side of the HEU's 22 may becontrolled in substantially independent manners (e.g., by operation athree-way valves (not shown)). In one regard, the flow of cooling fluidthrough certain HEU's 22 may be restricted or substantially halted, forinstance, in situations where the computer systems for which an HEU 22is configured to cool are in idle states. It should be understood thatthe configuration of HEU's 22, pumps 38, fluid lines 28, and valves 40depicted in FIG. 3 is but one manner of a variety of differentobtainable configurations. It should also be understood that thespecific configuration for any given room may depend upon a multitude offactors, e.g., cooling needs, room configuration, cooling device type,etc. In addition, the fluid lines 28 may be positioned above the fans 30such that air may substantially be cooled prior to flowing into the fans30 without deviating from the scope of the present invention.

[0043]FIG. 4 is an exemplary block diagram 50 for a cooling system 52according to an embodiment of the invention. It should be understoodthat the following description of the block diagram 50 is but one mannerof a variety of different manners in which such a cooling system 52 maybe operated. In addition, it should be understood that the coolingsystem 52 may include additional components and that some of thecomponents described may be removed and/or modified without departingfrom the scope of the invention.

[0044] The cooling system 52 includes an HEU controller 54 configured tocontrol the operations of the HEU 56. The HEU controller 54 may comprisea microprocessor, a micro-controller, an application specific integratedcircuit (ASIC), and the like. The HEU controller 54 may, for example,control the speed of the fans 58, the operation of the pump 60, thevalve 62, and the pipe valve 78. Interface electronics 64 may beprovided to act as a control interface between the HEU controller 54 andthe HEU 56, pump 60, and valve 62. Alternatively, a control interface mybe omitted between the HEU controller 54 and the above-describedcomponents as seen with respect to the connection between the HEUcontroller 54 and, for example, the pipe valve 78. The HEU controller 54may also be interfaced with a temperature sensor 68, e.g., one or moreof the temperature sensors described hereinabove with respect to FIG. 2.

[0045] The HEU controller 54 may be interfaced with an HEU memory 66configured to provide storage of a computer software that provides thefunctionality of the HEU controller 54, e.g., the speed of the fans,pump operation, etc. The HEU memory 66 may be implemented as acombination of volatile and non-volatile memory, such as dynamic randomaccess memory (DRAM), EEPROM, flash memory, and the like. The HEU memory66 may also be configured to provide a storage for containing dataand/or information pertaining to the manner in which HEU controller 54may operate the fans 58, the pump 60, the valve 62, and the pipe valve78. In one respect, the manner of operation of the above-describedcomponents may be based according to temperature measurements by thetemperature sensor 68.

[0046] The HEU controller 54 may be further interfaced with a coolingdevice controller 70. The interface may be effectuated via a wiredprotocol, such as IEEE 802.3, etc., wireless protocols, such as IEEE801.11b, wireless serial connection, Bluetooth, etc., or combinationsthereof.

[0047] The cooling device controller 70 may be configured to control theoperations of the cooling device 72. The cooling device controller 70may comprise a microprocessor, a micro-controller, an applicationspecific integrated circuit (ASIC), and the like. The cooling devicecontroller 70 is generally configured to manipulate the temperature ofthe cooling fluid by controlling the operation of the cooling device 72.In this regard, the cooling device 72 may comprise a variable speedcompressor, a heat exchanger, a chilled water heat exchanger, acentrifugal chiller, and the like. More particularly, the cooling devicecontroller 70 may be designed to vary the operation of one or more ofthe above-recited components to vary the amount of heat transfer on therefrigerant contained in the refrigeration loop of the cooling device 72to thereby vary the cooling fluid temperature.

[0048] Interface electronics (I/F) 74 may be provided to act as aninterface between the cooling device controller 70 and the componentsfor operating the cooling device 72, e.g., the supply of voltage to varythe speed of the compressor, control of the heat exchanger (centrifugalchiller) capacity, etc.

[0049] The cooling device controller 70 may also be interfaced with acooling device (CD) memory 76 configured to provide storage of acomputer software that provides the functionality of the cooling device72, e.g., compressor, heat exchanger, and the like, and may be executedby the cooling device controller 70. The CD memory 76 may be implementedas a combination of volatile and non-volatile memory, such as DRAM,EEPROM, flash memory, and the like. The CD memory 76 may also beconfigured to provide a storage for containing data/informationpertaining to the manner in which the compressor (heat exchanger,chiller) may be manipulated in response to, for example, variations inthe temperature of the cooling fluid and/or air in the room.

[0050] In one respect, the capacity (e.g., the amount of work exerted onthe cooling fluid) of the compressor (heat exchanger, chiller, etc.) maybe modified to thereby control the temperature and of the cooling fluid.The compressor (heat exchanger, chiller, etc.) may thus be controlled toeither increase or decrease the mass flow rate of the refrigerantflowing therethrough. Consequently, when the temperature in the room 10is below a predetermined range, the capacity of the (heat exchanger,chiller, etc.) may be reduced to substantially reduce the amount ofwork, thus the amount of energy exerted on the refrigerant. This maylead to a significant reduction in the costs associated with operatingthe cooling system 52 as compared with conventional cooling systems.

[0051] Because the specific type of compressor (heat exchanger, chiller,etc.) to be employed with embodiments of the invention may varyaccording to individual needs, the invention is not limited to anyspecific type of compressor(heat exchanger, chiller, etc.). Instead, anyreasonably suitable type of compressor (heat exchanger, chiller, etc.)capable of accomplishing certain aspects of the invention may beemployed with the embodiments of the invention. The choice of compressor(heat exchanger, chiller, etc.) may therefore depend upon a plurality offactors, e.g., cooling requirements, costs, operating expenses, etc.

[0052] By way of example, if there is a detected or anticipated changein the temperature of a rack, e.g., rack 12, areas surrounding the rack,etc., the HEU controller 54 may operate to manipulate the correspondingHEU 56 and/or fan(s) 58 to compensate, e.g., change the volume flowrate, velocity, and other characteristic of the air flow, for the changein temperature. In addition, or as an alternative to the above, thecooling device controller 70 may operate to cause the cooling device 72to lower the refrigerant temperature. In this respect, each of the racks12 and/or portions thereof may generally receive substantially only theamount of cooling air necessary to maintain the temperature of the racks12 within a predetermined temperature range. In addition, therefrigerant, and therefore the cooling fluid, temperature may also becontrolled as needed to substantially optimize cooling of the racks 12.By controlling the cooling air flow in the above-described manner, thecooling system 52 may be operated at substantially optimized levels,thereby decreasing the amount of energy and thus the operating costsrequired to cool the computer systems in the racks 12.

[0053] Moreover, the cooling device controller 70 may operate thecompressor (heat exchanger, chiller, etc.) in a manner to vary thecooling fluid temperature in response to various degrees of detectedincreases/decreases in the operation of the HEU 56. More particularly, alook up table (not shown) may be stored in the CD memory 76. The look uptable may include information pertaining to the level of compressorspeed (heat exchanger capacity, etc.) increase necessary for a detectedincrease in the operation of the HEU 56. In this respect, the compressorspeed (heat exchanger capacity, etc.) may be varied substantiallyincrementally in response to detected changes in the operation of theHEU 56.

[0054]FIG. 5 shows a flow diagram of an operational mode 100 accordingto an embodiment of the invention. It should be understood that theoperational mode 100 may include additional operations and that some ofthe operations may be removed and/or modified without departing from thescope of the invention. The following description of the operationalmode 100 is made with reference to the block diagram 50 illustrated inFIG. 4, and thus makes reference to the elements cited therein.

[0055] The operations illustrated in the operational mode 100 may becontained as a utility, program, or a subprogram, in any desiredcomputer accessible medium. In addition, the operational mode 100 may beembodied by a computer program, which can exist in a variety of formsboth active and inactive. For example, they can exist as softwareprogram(s) comprised of program instructions in source code, objectcode, executable code or other formats. Any of the above can be embodiedon a computer readable medium, which include storage devices andsignals, in compressed or uncompressed form.

[0056] Exemplary computer readable storage devices include conventionalcomputer system RAM, ROM, EPROM, EEPROM, and magnetic or optical disksor tapes. Exemplary computer readable signals, whether modulated using acarrier or not, are signals that a computer system hosting or runningthe computer program can be configured to access, including signalsdownloaded through the Internet or other networks. Concrete examples ofthe foregoing include distribution of the programs on a CD ROM or viaInternet download. In a sense, the Internet itself, as an abstractentity, is a computer readable medium. The same is true of computernetworks in general. It is therefore to be understood that thosefunctions enumerated below may be performed by any electronic devicecapable of executing the above-described functions.

[0057] In the operational mode 100, the cooling system 52 is activatedat step 110. At step 120, the temperature of one or more racks (Tr) issensed by temperature sensors, e.g., one or more temperature sensors 68.The temperature sensors may comprise one or more of the above-describedtemperature sensors, e.g., thermocouple, mobile environmental conditionsensing device, etc. The Tr's generally correspond to the heat load ofthe heat dissipating components and therefore the computer systemscontained in the racks, e.g., rack 12. Therefore, the Tr's may be basedupon the temperatures of specific heat dissipating components andsubsystems. In addition, the Tr's may be based upon the temperatures inthe general vicinity of the racks and/or sections of the racks. Thus,those skilled in the art will understand that certain embodiments of thepresent invention may be employed with temperature sensors located atvarious positions throughout the room, e.g., room 10. Moreover, use ofthe term “rack” herein generally refers to racks for supporting computersystems and additionally to sections of the racks as well as areasaround the racks. Therefore, the use of the term “rack” throughout thepresent disclosure is not meant to limit certain aspects thereof toentire racks, but instead, is relied upon to simplify the description ofcertain embodiments of the present invention.

[0058] As an alternative to the temperature sensor 68, the Tr's may beanticipated in the manner described hereinabove and in co-pending U.S.patent application Ser. No. 09/970,707.

[0059] At step 130, it is determined whether each of the Tr's is withina predetermined range of operating temperatures, e.g., between a maximumset point temperature (Tmax,set) and a minimum set point temperature(Tmin,set). In general, the range of temperatures Tmin,set and Tmax,setpertains to threshold temperatures to determine whether to increase ordecrease the flow of cooling air delivered to the racks. This range ofoperating temperatures may be set according to a plurality of factors.These factors may include, for example, the operating temperatures setforth by the manufacturers of the subsystems and components located inthe racks, through testing to determine the optimal operatingtemperatures, etc. In addition, the predetermined range of operatingtemperatures may vary from one subsystem to another.

[0060] For those racks having Tr's that are within the predeterminedrange, their temperatures are sensed again at step 120. For those racksdetermined to have heat loads that do not fall within the predeterminedtemperature range, i.e., fall outside of Tmin,set and Tmax,set, the HEUcontroller 54 may determine whether those racks have temperatures thatare below the Tmin,set at step 140.

[0061] The air flow supplied by the HEU 56, and more particularly thefans 58, configured to supply cooling air flow to the racks having Tr'sbelow or equal to the Tmin,set, may be decreased at step 150. Inaddition, and/or alternatively, the temperature of the cooling fluidsupplied through the HEU's 56 may be increased to thereby increase thetemperature of the air supplied to these racks. As describedhereinabove, this may be accomplished by decreasing the work placed onthe refrigerant by the cooling device 72 and/or by opening the bypassvalve 62. Additionally, the temperature of the air flowing from the HEU56 may also be increased by causing the valve 78 to enable a lowervolume flow rate of cooling fluid to flow therethrough.

[0062] The air flow supplied by the HEU 56 configured to supply coolingair flow to the racks having Tr's above the Tmin,set, and thus above theTmax,set, may be increased at step 160. In this regard, the HEUcontroller 54 may increase power to the fans 58 to enable greater airflow to the racks. In addition, and/or alternatively, the HEU controller54 may increase the flow of cooling fluid through the HEU 56 viaoperation of the pump 60 and/or the valve 78.

[0063] By virtue of certain embodiments of the invention, racks havingvarious Tr's may be supplied with air flow having variouscharacteristics, e.g., temperature, velocity, direction, and the like.That is, for example, in a system comprising a plurality of HEU's 56,air flow may be supplied to racks having higher Tr's by certain HEU's 56substantially simultaneously with air flow supplied to racks havingrelatively lower Tr's by other HEU's 56 where each of the racks receivesdifferent levels of air flow. Additionally, the temperature of thecooling fluid supplied through the HEU's 56 delivering air flow to rackshaving higher Tr's may be relatively lower than the temperature of thecooling fluid supplied through the HEU's 56 delivering air flow to rackshaving lower Tr's. It should thus be appreciated that steps 150 and 160maybe respectively and substantially simultaneously performed by variousHEU's 56.

[0064] According to an embodiment of the invention, the decrease involume flow rate and/or velocity of the air flow at step 150 and theincrease in volume and/or velocity of the air flow at step 160 may beaccomplished by incrementally varying the air flow from the HEU 56. Forexample, the volume flow rate and/or velocity of the air flow from theHEU 56 may be varied by a predetermined amount during a first iteration.The Tr may be measured at a predetermined time following the variationand steps 130-160 may be repeated. By repeating this process a number oftimes, the temperature of the rack may be substantially brought withinthe predetermined range.

[0065] At step 170, the cooling device controller 70 may determinewhether to vary the cooling fluid temperature, e.g., increase thecompressor speed (heat exchanger capacity, etc.). The cooling fluidtemperature may be decreased in response to, for example, actual and/ordetected increases in Tr. In the alternative, the decrease in coolingfluid temperature may be predicated upon the amount of work performed bythe HEU 56. More particularly, as the work performed by the HEU 56increases, thereby signaling an increase in the Tr, the cooling devicecontroller 70 may cause the cooling device 72 to decrease therefrigerant temperature. In addition, when a plurality of HEU's 56 areimplemented to cool one or more racks, the cooling device controller 70may determine whether the total amount of decreases in the volume flowrates of the cooling air exceeds the total amount of increases in thevolume flow rates flow of the cooling air. At step 180, the coolingdevice controller 70 may determine whether to decrease the cooling fluidtemperature.

[0066] At step 190, the cooling device controller 70 may cause thecooling fluid temperature to be decreased in response to a determinationto do so at step 180. Alternatively, the cooling device controller 70may cause the cooling fluid temperature to be increased at step 200 inresponse to a determination to do so at step 180. As describedhereinabove, the cooling device controller 70 may operate to vary thetemperature of the cooling fluid by varying the speed of the compressor,capacity of the heat exchanger, and the like.

[0067] It should be understood that steps 150, 160, and 190 or 200 maybe performed substantially simultaneously. More particularly, on a zonallevel, temperature readings determined at step 120 for various racks maybe implemented in varying the delivery of cooling air to the variousracks. Additionally, on a global level, the temperature of the coolingfluid may be varied according to the changes in the cooling requirementsof the racks. For example, when the temperature of a rack exceeds apredetermined threshold range, the HEU(s) 56 supplying cooling air flowto that rack may increase the air flow to that rack. Substantiallyconcurrently, the temperature of the cooling fluid may be decreased byan amount relative to the increased rack temperature.

[0068] Following steps 190 or 200, the Tr's are sensed again at step120. In addition, the steps following step 120 may be repeated for anindefinite period of time and may form a closed-loop pattern.

[0069] In accordance with embodiments of the present invention, thecooling requirements of a room configured to house computer systems maybe analyzed to substantially optimize the layout of the racks within theroom. In one respect, the substantial optimization of the rack layout inthe room may enable the cooling system of the data center to operate atgenerally lower energy and greater efficiency levels by virtue of thereduced workload placed on the components of the cooling systems, e.g.,compressors, HEU's, etc. In addition, the cooling requirements may beanalyzed to optimize the workload of the computer systems as describedabove. In this regard, the actual or anticipated temperatures may bestored in a look-up table and used as references in optimizing the roomlayout and/or workload distribution. The cooling requirements within theroom may be analyzed by operation of any reasonably suitablecommercially available computational fluid dynamics (CFD) tool, e.g.,FLOVENT, a 3-D modeling software capable of predicting temperaturevariations based upon fluid flow analyses.

[0070] By virtue of the numerical modeling, various cooling devices aswell as the HEU's described hereinabove may be positioned throughout theroom to substantially control the manner in which the racks receive thecooling fluid.

[0071] In addition, the cooling devices may also be positioned tosubstantially maximize and optimize their performances, e.g., to preventone or more of the cooling devices from operating at substantiallyexcessive levels.

[0072] In determining the cooling air distribution requirement withinthe room, each of the racks may be assigned a heat load which maycorrespond to a maximum heat load predicted for that rack, e.g., throughanticipated power draw. For example, a rack containing 40 subsystems,e.g., computers, may have a maximum heat load of 10 KW and a rackcontaining 20 subsystems may have a maximum heat load of 5 KW. Byimplementing the CFD in this manner, for example in a room containing100 racks, four cooling devices, and a plurality of HEU's, racks havinga potential for relatively larger heat loads may be relativelyseparately located throughout the data center. In one respect,therefore, the cooling devices and the HEU's within the room may beoperated at substantially less than maximum power levels while the racksmay receive sufficient amounts of cooling fluid. More specifically, thepower required to operate the cooling devices and the HEU's may beregulated to efficiently supply cooling fluid to the racks by providingsubstantially only that amount of cooling fluid necessary to maintainthe racks within nominal operating temperatures.

[0073] Additionally, a CFD tool may be implemented substantiallysimultaneously with the cooling system. More specifically, the CFD toolmay be utilized to substantially continuously vary the operation of thecooling system to operate according to the heat loads generated in theracks. In this regard, the anticipated or actual heat loads (e.g., basedupon the power draw of the components) on the racks may be inputted intothe CFD tool, along with one or more of the following properties:velocity of the cooling fluid flowing through various sections of theroom and the distribution of temperature and pressure of the coolingfluid in the data center, to determine an optimal manner in which, forexample, the HEU's may be operated. The velocity of the air flow as wellas other atmospheric conditions at various locations within the room maybe sensed by a temperature sensor. The sensed conditions may betransmitted or otherwise relayed to the CFD tool to enable the tool toperform the necessary calculations.

[0074] In this regard, the CFD tool may be implemented to produce anumerical model of the room to thus determine an optimized coolingdistribution within the room. A correlation of one or more of thefollowing properties: velocity of the cooling fluid flowing throughvarious sections of the room, distribution of temperature and pressureof the cooling fluid in the room, and the power draw into the racks, maybe created based on the numerical modeling. The correlation may be usedto infer thermal conditions throughout the room when only a minimumnumber of sensors are available during operation of the cooling system.In addition, the correlation may substantially reduce the amount of timerequired for the CFD tool to perform the computing operations.

[0075] By virtue of certain embodiments of the present invention, one ofordinary skill in the art will readily recognize that the amount ofenergy, and thus the costs associated with cooling the racks locatedwithin a room may be substantially reduced. In one respect, by operatingthe cooling system to supply cooling fluid substantially only as neededby the racks, the cooling system may be operated at a relatively moreefficient manner in comparison with conventional cooling systems.

[0076] What has been described and illustrated herein is a preferredembodiment of the invention along with some of its variations. Theterms, descriptions and figures used herein are set forth by way ofillustration only and are not meant as limitations. Those skilled in theart will recognize that many variations are possible within the spiritand scope of the invention, which is intended to be defined by thefollowing claims—and their equivalents—in which all terms are meant intheir broadest reasonable sense unless otherwise indicated.

What is claimed is:
 1. A method for cooling a room configured to house aplurality of computer systems, said method comprising: providing a heatexchanger unit configured to receive air from said room and to deliverair to said room; supplying said heat exchanger unit with cooling fluid,wherein said cooling fluid is operable to cool said received air in saidheat exchanger unit; sensing temperatures at one or more locations insaid room; controlling at least one of the temperature of said coolingfluid and said air delivery to said room in response to said sensedtemperatures at said one or more locations.
 2. The method according toclaim 1, further comprising: providing a cooling device configured tomanipulate the temperature of said cooling fluid, wherein said step ofcontrolling at least one of a temperature of said cooling fluid and saidair delivery to said room comprises varying an output of said coolingdevice to control the temperature of said cooling fluid.
 3. The methodaccording to claim 2, wherein said cooling device comprises arefrigeration loop having a variable capacity compressor and arefrigerant, and wherein said step of controlling the temperature ofsaid cooling fluid comprises controlling the temperature of saidrefrigerant through operation of said variable capacity compressor. 4.The method according to claim 1, wherein said heat exchanger unitcomprises at least one fan configured to effectuate delivery of the air,and wherein said step of controlling at least one of a temperature ofsaid cooling fluid and said air delivery to said room comprises varyingan output of said fan to control the delivery of air to the room.
 5. Themethod according to claim 1, further comprising: determining whether thesensed temperatures at one or more locations in said room are within apredetermined range.
 6. The method according to claim 5, furthercomprising: determining whether the sensed temperatures is at least oneof less than and equal to a minimum set point temperature in response tosaid sensed temperatures at one or more locations in said room beingoutside of said predetermined range.
 7. The method according to claim 6,wherein said controlling step comprises decreasing said air delivery tosaid room in response to said sensed temperatures at said one or morelocations being at least one of less than and equal to a minimum setpoint temperature.
 8. The method according to claim 6, wherein saidcontrolling step comprises increasing said air delivery to said room inresponse to said sensed temperatures being above said minimum set pointtemperature and outside of said predetermined range.
 9. The methodaccording to claim 5, further comprising: varying the cooling fluidtemperature in response to the sensed temperatures at one or morelocations in said room being outside of said predetermined range. 10.The method according to claim 9, further comprising: increasing saidcooling fluid temperature in response to a sum of the sensedtemperatures at one or more locations being below said predeterminedrange.
 11. The method according to claim 9, further comprising:decreasing said cooling fluid temperature in response to a sum of thesensed temperatures at one or more locations being above saidpredetermined range.
 12. The method according to claim 1, furthercomprising: performing a numerical modeling of a temperaturedistribution and flow characteristics of air within the room; andmanipulating said cooling system in response to said numerical modeling.13. The method according to claim 12, further comprising: implementingsaid numerical modeling to correlate at least two of temperature,velocity and pressure of said cooling fluid and power draw of said rackswithin said data center to thereby infer a thermal condition throughoutsaid room, wherein said controlling step further comprises manipulatingsaid cooling system in response to said inferred thermal condition. 14.The method according to claim 1, wherein said step of sensingtemperatures at one or more locations in said room comprises sensingsaid temperatures with a mobile environmental condition sensing device.15. The method according to claim 1, wherein said step of sensingtemperatures at one or more locations in said room comprises determiningsaid temperatures according to anticipated heat loads of one or more ofsaid plurality of computer systems.
 16. The method according to claim 1,further comprising: manipulating the workload on the plurality ofcomputer systems to optimize energy efficiency in cooling said pluralityof computer systems.
 17. The method according to claim 1, wherein saidstep of controlling at least one of the temperature of said coolingfluid and said air delivery to said room in response to said sensedtemperatures at said one or more locations comprises substantiallyceasing air delivery and cooling fluid delivery in response to saidsensed temperatures being below a predetermined set point temperature.18. A system for cooling a room containing one or more computer systems,said system comprising: a heat exchanger unit configured to receivecooling fluid through a cooling fluid line and to receive air, whereinsaid air may be cooled through heat transfer with said cooling fluid;said heat exchanger unit having at least one fan configured to cause airto flow out of the heat exchanger unit; a heat exchanger controlleroperable to control a supply of said cooling fluid to said heatexchanger unit and operable to control the speed of the at least onefan; a device for cooling said cooling fluid; and a cooling devicecontroller configured to operate the device for cooling to vary thetemperature of said cooling fluid.
 19. The system according to claim 18,further comprising one or more temperature sensors, wherein said heatexchanger controller is configured to receive environmental conditioninformation from said one or more temperature sensors.
 20. The systemaccording to claim 19, wherein said temperature sensor comprises atleast one of a thermocouple and a mobile environmental condition sensingdevice.
 21. The system according to claim 18, wherein said computersystems are housed in a plurality of racks, and wherein a plurality ofheat exchanger units are positioned at various locations in said room tosupply air flow to said plurality of racks in a substantiallyindependent manner.
 22. The system according to claim 21, furthercomprising: one or more pumps configured to control cooling fluiddelivery to one or more of said plurality of heat exchanger units,wherein said heat exchanger controller is operable to control said oneor more pumps.
 23. The system according to claim 18, further comprising:a valve configured to meter the flow of cooling fluid through said heatexchanger positioned along said cooling fluid line generally upstream ofsaid heat exchanger unit, wherein said heat exchanger controller isoperable to control the mass flow rate of said cooling fluid throughsaid valve.
 24. The system according to claim 18, wherein said coolingdevice comprises at least one of a variable capacity compressor, a heatexchanger, a chiller, and a cooling device controller configured tocontrol said at least one of said variable capacity compressor, saidheat exchanger, and said chiller.
 25. The system according to claim 24,wherein said cooling device controller is configured to communicate withsaid heat exchanger controller.
 26. The system according to claim 25,wherein said communication between the cooling device controller and theheat exchanger controller includes communication of informationpertaining to the level of operation of said one or more heat exchangerunits, and wherein said cooling device controller is configured tooperate said at least one of the variable capacity compressor, heatexchanger, and chiller in response to said information.
 27. The systemaccording to claim 25, wherein said communication comprises at least oneof temperature measurements and heat exchanger unit operations.
 28. Thesystem according to claim 18, wherein said heat exchanger unit issupported from a ceiling of said room.
 29. The system according to claim18, further comprising: a connecting line between a supply portion ofsaid cooling fluid line and a return portion of said cooling fluid line;and a valve configured to meter the flow of cooling fluid through saidconnecting line, wherein said heat exchanger controller is operable tocontrol the flow of said cooling fluid through said valve.
 30. A systemfor cooling computer systems housed in one or more racks, said racksbeing maintained in a room, said system comprising: means for coolingair including means for receiving cooling fluid; means for deliveringcooled air to said computer systems; means for measuring temperatures atone or more locations in said room; means for controlling delivery ofsaid cooled air in response to the temperature measurements; and meansfor controlling the temperature of said cooling fluid.
 31. The systemaccording to claim 30, further comprising: means for controllingdelivery of cooling fluid through said cooling means.